Validating generative AI in banking

When federal banking regulators issued SR 26-2 in April 2026, they replaced SR 11-7 with a modern, risk-based model risk framework and then did something striking: they placed generative AI explicitly outside its scope and promised a request for information to come. That leaves banks running large language models on consequential work, from drafting suspicious-activity reports to generating adverse-action notices, without an agreed validation standard. Our new paper closes that gap. It shows that the four enduring pillars of model risk management, conceptual soundness, ongoing monitoring, outcomes analysis, and effective challenge, carry over to generative systems intact, and it turns them into concrete tests: benchmarking against defensible ground truth, stability measured rather than assumed, outcome analysis that weighs errors by what they cost and checks consistency across the book, and a materiality tier that scales the effort to the risk. We prove the framework on a fully worked validation of an LLM SAR-drafting pipeline, where the system clears every headline benchmark yet still fails three tests that matter, a stability collapse at production temperature, a real performance gap on the highest-risk customer segment, and a silent model-version drift, none of which a benchmark alone would catch. Every test maps to specific SR 26-2 language, every number is reproducible from a fixed seed, and the result is a standard banks can adopt now and the agencies could build on. Read the paper here.

Previous
Previous

Human capital and machine intelligence

Next
Next

Who pays the tariffs?